Volume rendering often utilizes stochastic-based path tracing and requires many ray casting draw calls to achieve high visual fidelity free of artifacts. Due to processing limitations in modern graphics processing units (GPUs), real-time user interaction with a path tracing renderer may only generate and display incomplete images to the user with rendering artifacts. The rendering artifacts are described as high frequency noise or “salt and pepper,” and are the result of the low number of ray casting draw calls used to generate and display the images to the user. In a medical imaging setting, the incomplete images may reduce a user's ability to diagnose and/or review patient data. Thus, due to the processing limitations, volume renderings from path tracing, such as Monte Carlo volume renderings, are often generated before viewing, or a-priori. The resulting images are saved to memory for future use. Generating volume renderings in this manner shifts the rendering process from real-time path tracing rendering to real-time image-based rendering.
One image-based rendering technique is lightfield or Lumigraph rendering. Lightfield rendering generates one or more virtual views from previously rendered or photographically captured images or image data. Lightfield rendering recreates detailed views of real scenes and renders volumes with very subtle interactions between light and matter. Lightfield representations are generated using a simplified plenoptic function. A plenoptic function describes the radiance for every position and orientation in a region of space. The plenoptic function includes three dimensions describing position and two dimensions describing orientation, resulting in a five-dimensional function. Using a plenoptic function, every possible view within a scene may be reconstructed and rendered by sampling the plenoptic function for each ray generated by the camera. The five-dimensional plenoptic function may be prohibitively large in practice, and generation of the plenoptic function is not possible within occluded spaces. Therefore, simplifying assumptions can be made to the plenoptic function. For example, the plenoptic function is sampled outside the bounding hull of the rendered volume. Additionally, empty space around the volume, such as air, is assumed to not attenuate the light. Using these assumptions, the radiance along a ray stays is constant, and the two dimensions describing orientation can be reduced by one, resulting in a more manageable four-dimensional function for the lightfield rendering.
Lightfield image-based rendering has several drawbacks. Lightfield rendering techniques often employ computationally-expensive rendering algorithms used to generate the lightfield rendering. Further, the data size of the lightfield representation component images may be prohibitively large, and may be too large for a device's main or GPU memory all at once, and may be too large for contemporary mobile devices. The component images of the lightfield representation are also captured and generated at a single resolution and with limited camera rotation and panning. Thus, the single resolution images prevent the end user from zooming in and out of the lightfield rendering without introducing image distortion, such as upsampling artifacts occurring from linear interpolation upsampling. Thus, a lightfield rendering is restricted to a small amount of user-interaction for zoom, rotation and panning.